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Cindy E. Hmelo, Gerald S. Gotterer, and John D. Bransford2 



Medical schools are increasingly looking to case-based formats such as 
problem-based learning (PBL) for their medical students. However, the effects of 
PBL have not been adequately assessed for an informed decision. It has been argued 
that PBL improves lifelong learning, clinical reasoning, and basic science learning 
but the literature on the cognitive benefits of PBL has shown mixed results (e.g., 
Albanese and Mitchell, 1993). For example, on the National Board of Medical 
Examiners (NBME) Part I, a test of Basic Science knowledge, PBL students tend to 
score lower than students in traditional curricula but students in a PBL curriculum 
tend to perform slightly better on NBME tests of clinical reasoning. However, some 
of the traditional measures (e.g., board scores) used in earlier studies of PBL may not 
have been sufficiently sensitive to reveal cognitive effects such as changes in 
reasoning and self-directed learning (SDL) strategies. The goal of the present study 
was to develop measures that can assess specific reasoning and learning changes 
purported to be affected by the PBL curriculum. At the same time, we develop * 
evidence of the cognitive benefits of a PBL approach. 

Theoretical Framework for PBL 

PBL includes among its goals: 1) developing scientific understanding through 
clinical cases 2) developing clinical reasoning strategies, and 3) developing self- 
directed learning strategies. Research and theorizing in cognitive psychology 
provides evidence that supports some of the general goals of PBL (Norman, 1992). 
Additionally, laboratory-based research suggests that by anchoring instruction in 
authentic problem-solving contexts, the knowledge, clinical reasoning processes. 
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and SDL strategies learned should be more readily transferred when that knowledge 
or strategy is needed (e.g., Adams, Kasserman, Yearwood, Perfetto, Bransfbrd, and 
Franks, 1988; Williams, 1993). Reasons for the effectiveness of PBL include the fact 
that the relevance of science as a tool for vmderstanding and solving medical 
problems should be more readily apparent (Bransford, Franks, Vye, & Sherwood, 
1989). In addition, the acquisition of prior examples that occurs during PBL may 
allow later problems to be solved on the basis of similarity (Brooks, Norman, & 
Allen, 1991). FinaUy, the active learning promoted in PBL should promote the self- 
directed learning strategies and attitudes needed for Ufdong learning (Bereiter & 
Scardamalia, 1989; Ng & Bereiter, 1991). 

PBL and Clinical Reasoning 
In the current study, we examine whether PBL students reach the reasoning 
goals with a novel clinical case. We assess PBL students reasoning and integration 
of scientific information using paper-and-pencU instruments. We do this by 
evaluating: 1) directionaUty of feasoning, 2) coherence of explanations, and 3) use of 
basic science information in explanatiorw. 

Reasoning can be examined by looking at how the data and hypotheses are 
related to each other (see Figures la and lb). Data-driven reasoning involves 
reasoning from the data to a hypothesis whereas hypothesis-driven reasoning 
involves using a hypothesis to explain the data. An example of a data-driven 
reasoning statement is "If he has an elevated blood sugar, then he must have 
diabetes." "Because he has diabetes, he has an elevated blood sugar'' is an example 
of a hypothesis-driven reasoning statement. Other relations may also be expressed, 
for example, "He has an infection in association with diabetes." In this case, the 
directionaUty of reasoning is unclear. This may occur for several reasons. First, the 
problem-solver may not be clear about causaUty. Second, there may not be, in 
reality, a clear causal relationship. Subjects may also assert causes without justifying 
them. These are statements such as "He is diabetic" without any explanations to 
support the hypothesis. 
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Insert Figures la and lb about here 



Although data-driven reasoning is more characteristic of experts, it is 
inappropriate for novices who have an insufficient knowledge base (Patel and 
Kaufman, 1993). In PBL, students are taught to use hypothesis-driven reasor»ing to 
construct explanations that account for all of the data. Thus we predicted that the 
PBL students would be more likely to use hypothesis-driven reasoning than 
conventional (nonPBL) students because that is the strategy they were taught Prior 
research has demonstrated that traditional medical students were more likely to use 
data-driven reasoning although there is not a clear theoretical basis for this 
prediction (Patel, Groen, & Norman, 1993). 

The nature of hypothesis-driven reasoning allows students to learn to filter 
relevant from irrelevant information. One expected result is a coherent 
explanation. Cohere ice of explanation was assessed by looking at the maximum 
number of relational operators chained together in an explanation. For example, an 
explanation such as "because he is diabetic, he cannot metabolize carbohydrates so 
he must use ketone bodies for energy causing a metabolic acidosis" would be scored 
as containing three relations but "he has an acidosis which causes an increased 
respiratory rate" would be scored as containing one relation. We expected that the 
PBL students would generate longer reasoning chains than nonPBL students. In a 
study of radiologists, a similar measure of coherence revealed an effect of expertise 
(Lesgold, Rubinson, Feltovich, Glaser, Klopfer, & Wang, 1988). Senior radiologists 
had longer reasoning chains indicative of more coherent explanations than 
residents. This suggests that experts are doing more inferential thinking and ending 
up with a more coherent representation of the patient. Alternatively, novice's 
explanations, with shorter reasoning chains, suggest a more fragmented 
representation. 

Because PBL students are learning basic science in the context of clinical cases, it 
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is reasonable to expect them to use this information, in their explanations. We coded 
explanations for concepts or facts from anatomy, physiology, biochemistry, 
microbiology, or pathology. These provide evidence that students are integrating 
basic science information. In a recent study by Patel, Green, and Norman (1993), PBL 
and conventional students wei^ asked to construct causal explanations and integrate 
relevant basic science information into their think-aloud explanations. They found 
that the PBL students incorporated more of the basic science information into their 
explanations and generated mo.d hypothesis-driven explanations than the 
conventional students. The current work attempts to replicate these findings using 
a paper-and-pencil instrtiments. 

PBL and Self-Directed Learning 
In addition to reasoning strategies, the current study examines the impact of PBL 
on SDL. Self-directed learning has several components. First, learners must assess 
their own state of knowledge relative to the problems they face. They must 
formulate these learning needs so they can efficiently meet these needs and make 
use of appropriate resources. Blumberg and Michael (1991) have provided indirect 
evidence to show that a PBL curriculum ht3 an impact on self-directed learning; 
PBL students borrowed more materials from the library than conventional medical 
students. They did not tvaluate the self-assessment of learni-ig needs or the 
generation of learning plans. 

There is little empirical research that clearly defines the nature of good self- 
directed learning strategies (Hmelo, 1993). Theoretically, when one has a 
hypothesis, choosing to search -n a hypothesis-driven manner may be more efficient 
than a search that is data-driven. A search that proceeds by investigating the 
significance of isolated findings may not effectively narrow the search space unless 
of cotirse one hits on the correct pathognomic finding to look up.3 In a recent paper 
by Bassok and Holyoak (1993), a distinction is made between top-down and bottom- 

3 A pathognomic finJing is a characteristic that is unique to a particular disease. The presence 
of such a sign or symptom allows positive diagnosis of the disease. 



up learning. Bottom-up learning refers to inductive learning by examples. Top- 
down learning depends on prior knowledge of the domain coupled with active 
learning strategies that allow the learner to make principled judgements about the 
importance of features to the learner's goals. Bottom-up learning requires that 
students make generalizations from r:.ultiple examples. The learners do not engage 
in deep analysis of principles and may end up knowing sets of correlated features 
(including some that are irrelevant). To the extent that top-down learning enables 
learners to successfully identify relevant but nonobvious features of a problem, 
more flexible transfer will be promoted. If the domain-knowledge is fragmented 
however, students may need to have their attention directed to goal-relevant 
aspects. In problem-based learning the students are encouraged to think about the 
cases with the underlying scientific principles in mind rather than just collecting 
sets of features. As cases are connected to domain principles, the learner can begin 
to understand how knowledge can be applied to solving problems (Chi, Bassok, 
Reimann, Lewis, & Glaser, 1989). Self-directed learning that proceeds in a 
hypothesis-driven manner (i.e., top-down) should lead to more flexible knowledge. 

In this study, we also examine whether PBL students can assess their learning 
needs and develop a plan to address those needs. The learning needs (i.e., learning 
issues) fall into three categories: disease-driven, data-driven, and basic-science 
issues. An example of a disease-driven learning issue is "complications of diabetes.'' 
Disease-driven learning issues map onto hypothesis-driven reasoning in that they 
indicate the students are researching hypothesized disease mechanisms to try to 
account for the data. A basic-science driven learning issue might be "add-base 
physiology." An example of a data-driven learning issue is "the significance of an 
elevated respiratory rate/' Data-driven learning issues map onto data-driven 
reasoning because the individual is researching the pattern of data in order to 
detern::jne the causes. If the students' SDL strategies are consistent with their 
reasoning strategies, the PBL students should generate more disease^driven and 
basic science learning issues whereas the nonPBL students should generate more 
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data-driven learning issues. 

We would predict that given their experience, the PBL students would be more 
facile with choosing the learning resources for this task. However, the nonPBL 
students also have considerable experience in writing papers and using the library. 
We make no predictions about which group would use more resources but we do 
expect them to use different resources. The learning resources that the students 
used are categorized into basic science textbooks, clinical textbooks (which include 
both medical texts and diagnostic manuals), and use of expert consultants. We 
expected more basic science textbooks to be used by the PBL students because that 
maps onto the hypothesis-driven reasoning, integration of science information and 
the associated learning issues. We expected more clinical textbooks to be used by the 
nonPBL students because that maps onto data-driven reasoning. 

Methods 

Forty medical students participated in this study. Twenty students had 
participated in an elective class in PBL; twenty students (the nonPBL group) had 
taken a different elective. Within each condition, half the students were in their 
second year of medical school and half the students were three months into their 
first year of medical school. Each cohort of students was also taking the same 
traditional core curriculum. 

Instruction for the PBL students consisted of a group of eight medical students 
and a facilitator meeting for two hours, once each week. The facilitator gave the 
students a small amomat of information about the patient case. The group's task was 
to evaluate and define different aspects of the problem and to gain insight into the 
underlying causes of the disease process. This was done by questioning the 
facilitator, generating and evaluating hypotheses, and generating learning issues. 
Learning issues are topics that the group has decided are relevant and which they 
needed to learn more about. Hie group members would divide the learning issues 
and research them, using both material resources (e.g., the library) and expert 
consultants. During the next session, they then shared the information in an 
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attempt to explain the patient's disease process. 

To study the cognitive effects of instruction, students were presented with the 
case of a child with diabetic ketoacidosis. The case was presented in four segments: 
presenting complairJ, history, physical examination, and laboratory data. After each 
segment, the students were asked to explain, in writing, the underlying causal 
mechanisms that would account for the patient's problem. At the end of the case, 
the students were asked what they would want to learn more about to better 
understand the case and how they would go about meeting their learrung needs. 

Results 

The subjects' reasorung strategies, coherence, learrung self-assessments (referred 
to as learrung issues) and learrung plans were coded from the subjects written 
responses. These problem solving protocols were coded by a rater blind to 
conditions. A second blind rater independently scored a randomly selected subset of 
20% 01 the protocols. The two raters agreed on 92% of the responses. 
Problem-solving 

With respect to the reasorung strategies, the results indicated that PBL students 
were more likely to use hypothesis-driven reasorung in their explanations than 
nonPBL students, f (1,36)=15.59, p<.001, MSe=0.12 (Figure 2). In contrast, nonPBL 
students were more likely to use data-driven reasorung strategies (F(l,36)=55.23, 
p<.05,A4Se=0.06). Second-year nonPBL students were more likely than the second 
year PBL students to use other relations in their reasoning (F(l,36)=6.17; p<.05, 
MSej=0.05). This suggests that, in general, the PBL students transferred the 
hypothesis-driven reasorung strategies that they were taught. The second-year 
nonPBL students appear to realize that there are relationships between data and 
hypotheses but they do not make clear what these relatior\ships are. The groups did 
not differ on the number of unjustified assertions they made (Overall mean=:.60, 
F(U6)«1.78, p>.60). 




Insert Figure 2 about here 

By measuring the length of the students' reasoning chains^ differences in the 
coherence of the explanations can be detected. With this measure, we detected 
greater coherence in the PBL students' explanations. The PBL students generated 
longer reasoning chains than the nonPBL group as shown in Figure 3 (F(l,36)=7.22; 
p<.01, MSc=0.97). Finally; PBL students showed a marginal trend t^/ward using 
more basic science in their explanations however, this was a low frequency event 
(F(136)=:3.09, p<.10, MSc^OlO) see Figure 4). 

Insert Figures 3 and 4 about here 



Self-directed learning 

With respect to the SDL strategies, our measures revealed qualitative differences 
in the learning issues that PBL and nonPBL students generated. The students 
learning issues were categorized into, patterns that included disease-driven or basic- 
science Issues (combined to form hypothesis-driven patterns) and those that 
included data-driven or mixed (a combination of data-driven and 
hypothesis-driven) learning issues, xhe use of experts did not differ reliably across 
groups so this resource was not used in comparing the patterns of learning plans. 
As shown in Figure 5, the second-year PBL students generated exclusively 
hypothesis-driven patterns of learning issues whereas only 60% of second-year 
nonPBL students generated hypothesis-driven patt^^rns of learning issues (c2(l)=5.0, 
p<.05). Of the second-year nonPBLstudents, 40% included data-driven learning 
issues. For the first year students, the distribution of learning issues did not differ 
significantly between the two groups but the first year PBL students did show a trend 
toward more hypothesis -driven learning issues than their nonPBL counterparts 
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flexible kx^owledge and problem-solving. 

An alternate explanation for these results is that they were due to self-selection 
and preexisting differences. To deal with these limitations, additional research is 
needed that uses a longitudinal design. In addition, to generalize these results, 
further studies must use multiple cases and multiple sites, as well as to control for 
self-selection effeas». A study in progress focuses on using the this methodology in a 
longitudinal study designed to to compare the impact of fiill-time PBL, elective PBL, 
and conventional curricula. 

The current study, using an authentic diagnostic task, found that PBL instruction 
has distinct cognitive consequences that may influence medical practitioners 
throughout their careers. This is important as the medical education community 
decides whether to adopt a PBL approach. Prior studies have used traditional 
measures to assess PBL and have found few i-obust effects. Given that no strong 
differences between PBL and conventional students have been found on traditional 
measures such as board scores, the question becomes what are the subtler and more 
long-lasting effects that may not captured by traditional measures. In this study, we 
developed measures pertaining to the cognitive changes that affect lifelong learning. 
Our preliminary findings are important because they suggest that PBL may endow 
physicians with the learning strategies necessary to stay informed in the face of rapid 
medical advances. 
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(X2(l)=.83). 



Insert Figure 5 about here 



The PBL and nonPBL students generated different learning plans in a manner 
that was consistent with their learning issues and reasoning strategies (Figure 6). 
PBL students were more likely to use a combination of basic science and clinical texts 
whereas nonPBL students were more likely to use exclusively clinical texts 
especially at the first year level ixKl)=7.17, p<.05 for the first year students; 
X2(l)=1.50, p>.40 for the second year students). In fact, none of the first-year nonPBL 
students planned to use mixed resources. Recall that clinical texts include not only 
general medicine textbooks but diagnostic manuals as well. When the specific 
resources that students with data-driven learning plans were examined, it was 
found that for the first year nonPBL students, 40% of the resources in their plans 
were diagnostic texts compared with 22% of the first year PBL students' resources. 
So again, this pattern is consistent with the reasoning strategies the students were 
using. 



Insert Figure 6 about here 



Discussion 

The measures used in this study revealed a significantly greater use of 
hypothesis-driven reasoning in the PBL group. Moreover, by measuring the length 
of the reasoning chains, we detected a greater coherence in the PBL students' 
explanations. Our measures also revealed qualitative differences in the learning 
issues and plans that PBL and nonPBL students generated. These strategies were 
consistent with the reasoning strategies that the students used. The 
hypothesis-driven learmng strategies that the PBL students use should lead to more 
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Figure la. Data-driven Reasoning 
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Figure lb. Hypothesis-driven Reasoning 
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Figure 2. Directionality of Reasoning 
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Figure 3. Maximum Reasoning Chain Length 



Chain Length 




Year 2 Year 1 



Figure 4. Use of Basic Science Concepts 
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Figure 5. Self -directed Learning: Learning Issues 
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Figure 6. Self -directed Learning: Learning Plans 
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